Conditions 1 and 2

First let’s have a look at what is called Condition 1 and Condition 2 in the paper, i.e., in Experiment 1 the people who only saw five datapoints (always four red and one blue) and the ones who saw as many as they wanted; I’ve called these FIVE and UNLIMITED here.

Just so we can compare them directly, let’s put both posteriors on top of each other. We can see that as we would expect people tighten their generalisations with additional data.

Out of curiosity, let’s also see how many items people selected in the unlimited condition.

Condition 3 (ALL)

Now let’s take a look at the condition where we got priors, estimates after five, and estimates after an unlimited number; this was called Condition 3 in the paper and I’m calling it ALL here. Our first question is whether making them do the prior estimate made them much different from when they didn’t. So we want to compare five to five and unlimited to unlimited.

Each of these graphs show the data for Condition 3 (ALL), with the analogous data from the other ones shown as a dashed line and Xes.

These are virtually identical to the previous conditions (I’m not sure that correlation makes sense here but the graphs are pretty much superimposed on each other so I’m not worried. If we wanted to I suppose we could do t-tests at each point but that’s a lot of tests and it seems unnecessary).

Anyway though let’s also just sanity check that the number selected isn’t different across the two unlimited conditions.

These aren’t very different which we can verify with a t-test.

t.test(selected ~ condition, data=de1un_comp)
## 
##  Welch Two Sample t-test
## 
## data:  selected by condition
## t = 1.6513, df = 223.09, p-value = 0.1001
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.6056266  6.8677436
## sample estimates:
##       mean in group all mean in group unlimited 
##                21.17757                18.04651

This all suggests that people didn’t do weird things when they were asked to make estimates three times. So now let’s look at the overall priors they reported!

Overall it is sensible - mostly uniform, slightly peaked at 50%, no a priori preference for red or blue. But of course what we’re interested in is individual variation, not the overall.

Looking at fitted values: \(\beta\), \(\delta\), and \(\gamma\)

Let’s have a look at the distribution of parameters in Experiment 1. As a reminder:

\(\beta\): determines to what degree participants use their stated prior, and to what degree they use the uniform distribution to deduce their posterior. If \(\beta=1\) that means it’s all prior, if \(\beta=0\) it is all uniform distribution

\(\gamma\): how much they weighted the red chips they saw. if \(\gamma=1\) it is veridical, lower is underweighting, higher is overweighting.

\(\delta\): how much they weighted the blue chips they saw. if \(\delta=1\) it is veridical, lower is underweighting, higher is overweighting.

And also a 3D plot

Finally, let’s look at histograms of all of the variables.

Looking at fitted values: beta alone

I wanted to see if the beta values would be different if we weren’t also fitting delta and gamma. So let’s look at that histogram.

For the priors: 0% in FIVE and 0% in UNLIMITED had \(\beta\) less than 0.1, and 0% in FIVE and 0% in UNLIMITED had \(\beta\) greater than 0.9.

Looking at fitted values: \(\gamma\) alone

I wanted to see if the \(\gamma\) values would be different if we weren’t also fitting \(\delta\) and \(\beta\) So let’s look at that histogram.

For the likelihoods: 0% in FIVE and 0% in UNLIMITED had \(\gamma\) less than 1 (i.e., were conservative).

Looking at fitted values: \(\beta\) and \(\gamma\) alone

Here we assume one parameter (call it \(\gamma\)) instead of two separate ones (\(\gamma\) and \(\delta\)). i.e. this forces them to be the same and can be thought of as a conservatism parameter. I wanted to look at this because I think it might be a lot more interpretable than having both.

\(\beta\): determines to what degree participants use their stated prior, and to what degree they use the uniform distribution to deduce their posterior. If \(\beta=1\) that means it’s all prior, if \(\beta=0\) it is all uniform distribution

\(\gamma\): how much they weighted the chips they saw. if \(\gamma=1\) it is veridical, lower is underweighting, higher is overweighting.

Let’s calculate the Spearman correlation:

# five
cor.test(de1_fittwo$beta[de1_fittwo$section=="Fɪᴠᴇ"],
         de1_fittwo$gamma[de1_fittwo$section=="Fɪᴠᴇ"],
         method="spearman")
## Warning in cor.test.default(de1_fittwo$beta[de1_fittwo$section == "Fɪᴠᴇ"], :
## Cannot compute exact p-value with ties
## 
##  Spearman's rank correlation rho
## 
## data:  de1_fittwo$beta[de1_fittwo$section == "Fɪᴠᴇ"] and de1_fittwo$gamma[de1_fittwo$section == "Fɪᴠᴇ"]
## S = 195927, p-value = 0.6802
## alternative hypothesis: true rho is not equal to 0
## sample estimates:
##        rho 
## 0.04030897
# unlimited
cor.test(de1_fittwo$beta[de1_fittwo$section=="Uɴʟɪᴍɪᴛᴇᴅ"],
         de1_fittwo$gamma[de1_fittwo$section=="Uɴʟɪᴍɪᴛᴇᴅ"],
         method="spearman")
## Warning in cor.test.default(de1_fittwo$beta[de1_fittwo$section ==
## "Uɴʟɪᴍɪᴛᴇᴅ"], : Cannot compute exact p-value with ties
## 
##  Spearman's rank correlation rho
## 
## data:  de1_fittwo$beta[de1_fittwo$section == "Uɴʟɪᴍɪᴛᴇᴅ"] and de1_fittwo$gamma[de1_fittwo$section == "Uɴʟɪᴍɪᴛᴇᴅ"]
## S = 166778, p-value = 0.05908
## alternative hypothesis: true rho is not equal to 0
## sample estimates:
##       rho 
## 0.1830846

And a histogram of them too

For the priors: 51.4% in Fɪᴠᴇ and 56.1% in Uɴʟɪᴍɪᴛᴇᴅ had \(\beta\) less than 0.1, and 17.8% in Fɪᴠᴇ and 22.4% in Uɴʟɪᴍɪᴛᴇᴅ had \(\beta\) greater than 0.9.

For the likelihoods: 50.5% in Fɪᴠᴇ and 57.9% in Uɴʟɪᴍɪᴛᴇᴅ had \(\gamma\) less than 1 (i.e., were conservative).

Individuals: After five items

So now let’s look at individuals - compare their prior and posterior after five, based on the best-fit \(\beta\) and \(\gamma\)

red line: their reported prior

blue line: their reported posterior after five

dotted grey: Bayes rule prediction assuming uniform prior (not shown)

solid grey: Bayes rule prediction assuming their prior

dotted black: line based on best-fit \(\beta\) and \(\gamma\)

Let’s also look at this for the ones where we only fit \(\beta\).

And then with both \(\gamma\) and \(\delta\) as well as \(\beta\).

Individuals: After unlimited items

We’re going to do the same thing as before but this time comparing their prior and posterior after all of their items.

red line: their reported prior

dark blue line: their reported posterior after unlimited

dotted grey: Bayes rule prediction assuming uniform prior (not shown)

solid grey: Bayes rule prediction assuming their prior

dotted black: line based on best-fit \(\beta\) and \(\gamma\)

We can also get a sense of how good the fits were.

## TableGrob (1 x 2) "arrange": 2 grobs
##   z     cells    name           grob
## 1 1 (1-1,1-1) arrange gtable[layout]
## 2 2 (1-1,2-2) arrange gtable[layout]

Let’s also look at this for the ones where we only fit beta.

And then with both gamma and delta as well as beta.

Aggregate fits

We can also look at the parameter values for the aggregate fits. First, FIVE ITEMS

Then, UNLIMITED ITEMS